plantcomNGPN R
packageThe plantcomNGPN package was designed to help compile,
query and summarize plant community monitoring data collected by the
Northern Great Plains Inventory and Monitoring Network (NGPN) data and
stored in the FFI (FEAT/FIREMON Integrated) database.
There are a couple of ways to import NGPN FFI data into R, and which will be demonstrated in code below:
importData() function in this package can export a zip
file containing csvs of all of the raw FFI database tables. Once that
zip file is created, it can be used to import FFI data without the need
of SSMS or SQL Server Express. This option allows non-NPS users to work
with NPGN data. If this is the option you’re using, go to R package
setup tab for details on R package installation and setup.
importData() function compiles the FFI raw tables into
flattened stand-alone views of each FFI protocol (e.g. Cover Points,
Density Belts, Trees, etc.), which are used by package functions to
further query, summarize, and visualize the data. The views can be
exported as a zip file in the importData() function. The
zip file of views can be imported via importViews()
function. This option is the fastest way to import NPGN data into R, and
allows non-NPS users to more easily work with NPGN data. These views are
intended to be analysis-ready. Any feedback on how to improve their
usability is welcome. If this is the option you’re using, go to R
package setup tab for details on R package installation and setup.
Currently, only import and getter functions exist in the R package. In time, more features will be added, such as functions that perform common summaries and visualize data, and this tutorial will be updated accordingly. Feedback is always welcome on how to improve this tutorial and the R package in general.
Additionally, an automated QC report that checks NPGN FFI data for
potential missing data and errors is being developed separately.
Once this step is complete, users can import FFI database tables into
R using the importData() function and the name of the
database in SSMS (see ImportData tab).
To restore a database from a .bak file in SSMS, you can either restore through the file menu following the screencast or run the SQL code in SSMS below.
File Menu option:
SQL Code option: The SQL code below will restore a .bak file to a database named “FFI_RA_AGFO”. To use the code below, you only need to change “FFI_RA_AGFO” to match the name of your database (i.e., whatever precedes the .bak in the file name), and change the C:\temp\path to the path where your .bak file lives. Note that files on OneDrive often cause issues. It’s best to restore from locations directly on your C:\ drive or a server.
-- Variables to declare and modify for different database name and file locations
DECLARE @DBNAME NVARCHAR(MAX) = 'FFI_RA_AGFO'
DECLARE @FilePathOrig NVARCHAR(MAX) = 'C:\\temp\\'
DECLARE @FilePathNew NVARCHAR(MAX) = 'C:\\Program Files\\Microsoft SQL Server\\MSSQL15.SQLEXPRESS\\MSSQL\\DATA\\';
DECLARE @SQL NVARCHAR(MAX);
-- Run remaining query to restore database
--USE [master]
DECLARE @DATA NVARCHAR(MAX) = @FilePathNew + @DBNAME + '.mdf';
DECLARE @LOG NVARCHAR(MAX) = @FilePathNew + @DBNAME + '.ldf';
DECLARE @DBFULLPATH NVARCHAR(MAX) = @FilePathOrig + @DBNAME + '.bak';
DECLARE @DBLOG NVARCHAR(MAX) = @DBNAME + '_log';
USE[master]
SET @SQL =
' RESTORE DATABASE ['+@DBNAME+'] FROM DISK = '''+@DBFULLPATH+''' WITH FILE = 1, MOVE '''+@DBNAME+''' TO '''+@DATA+''',
MOVE '''+@DBLOG+''' TO '''+@LOG+''',
NOUNLOAD, STATS = 5'
EXEC(@SQL)
GO
…hopefully
If you’ve never installed an R package from GitHub, you’ll need to complete the following steps to install the plantcomNGPN R package. Once completed, move on to the ImportData and other function tabs for instructions on using the R package.
RTools is needed to install R packages from GitHub via devtools, and it only works with R versions 4.4.x. While R 4.5 is available on Software Center, the matching RTools45 isn’t available yet. Until that changes, link RStudio to the latest version of R 4.4 (I’m currently using R 4.4.3).
usethis line), so keep it handy. You
know when you have to rerun the code when you try to rebuild a package,
and a window pops up to ask if you want to install missing build files.
Install usethis and open .Renviron: If you don’t have the
usethis package, install it first (first line below). Once
installed, open the .Renviron using the second line below.
install.packages('usethis')
usethis::edit_r_environ()
Set RTools path: Next you’re going to tell .Renviron where to find RTools upon RStudio opening by pasting the code below into that file, and saving it. Now close/reopen RStudio, and you should be all set.
Sys.setenv(PATH = paste("C:\\PROGRA~1\\Rtools44\\bin", Sys.getenv("PATH"), sep=";"))
Sys.setenv(BINPREF = "C:\\PROGRA~1\\Rtools44\\mingw_$(WIN)\\bin\\")
install.packages('devtools')
To install R packages using devtools, you need to have a
GitHub user account and active token. Instructions for setting up a
GitHub account and connecting it to RStudio are on the
IMD
Advanced R training website from 2022. Go to Day 4: Version Control
> Git and RStudio and complete the steps described. There’s a lot of
other good info on using GitHub in other tabs too.
Note that whenever the plantcomNGPN package is updated,
you can rerun this code to install the latest version.
library(devtools)
install_github("KateMMiller/plantcomNGPN")
If you don’t want to have a GitHub account, you can also install packages from GitHub by downloading a .zip file of the code (often called forking a repo), then running the code below.
install.packages("plantcomNGPN.zip")
If you’re unable to install the R package via GitHub (often an error about permission being denied, or cannot open URL, download the following script from my OneDrive and open it in R: fix_TLS_inspection.R. Download by left clicking on the link, then click in the download arrow in the top left of the screen. Save it to your machine, then open it in RStudio.
Once this script is open in R Studio, press Control + A to select all of the code. Then Control + Enter to run all of the code. Assuming you don’t return any errors, you should be able to install from GitHub. Now try to reinstallplantcomNGPN. If you’re still running into
issues, it could be that devtools is missing some package
dependencies, which the error message will often mention. Install any
missing packages and then try installing plantcomNGPN again. If you’re
still not successful, send Kate Miller (kathryn_miller@nps.gov) a screenshot of the error and
she’ll help you troubleshoot.
library(plantcomNGPN)
importData() function imports FFI tables
and only returns the flattened views as data frames in an environment
called VIEWS_NGPN. This minimizes memory usage and keeps a tidy R
workspace. R package functions are then designed to work seamlessly with
the views, regardless of whether they’re in the VIEWS_NGPN environment
or global environment. See Working with the data tab for how to
work with data.frames housed within the VIEWS_NGPN environment.
Import data for 1 database using local install in SSMS
Note that R is not able to connect to files on Sharepoint or MS Teams (Teams also stores all files on Sharepoint). That means you need to store data package files on your local machine or on a server. The default option for importing data will add the data package views (i.e., flatfiles) to an environment called FFI_tables to your Environment work space (i.e. Environment tab in top right panel).
library(plantcomNGPN)
importData(dbname = "FFI_RA_AGFO")
Import data for multiple databases using local installs in SSMS
The code below will bind like data/tables across parks together to have 1 large view
importData(type = 'local',
dbname = c("FFI_RA_AGFO", "FFI_RA_BADL", "FFI_RA_DETO", "FFI_RA_FOLA",
"FFI_RA_FOUS", "FFI_RA_JECA", "FFI_RA_KNRI", "FFI_RA_MNRR",
"FFI_RA_MORU", "FFI_RA_SCBL", "FFI_RA_THRO", "FFI_RA_WICA"))
Import FFI tables as zip
The code below will import a zip file of THRO data exported by the
importData() function (see options below). This approach
does not require SSMS to be installed.
importData(type = 'csv', import_path = "C:/temp/FFI_RA_THRO.zip")
Import FFI views as a zip
The importViews() function is the fastest and least
memory hungry way to import FFI data. Someone will have to have
generated the views by first importing raw FFI tables, and then
exporting the views. Once that’s completed, this import function is
recommended.
importViews(import_path = "C:/temp/NGPN_FFI_views_20250710.zip")
Export FFI tables to zip
If you want to export the raw tables into a zip file of csvs, simply
add export_tables = T. If no export_path is specified, the
zip file will be saved in your working directory. The file name and path
of the zip file will be printed in your console. Note that exporting can
be slow, especially if you loaded multiple databases.
importData(type = 'local',
dbname = c("FFI_RA_AGFO", "FFI_RA_BADL", "FFI_RA_DETO", "FFI_RA_FOLA",
"FFI_RA_FOUS", "FFI_RA_JECA", "FFI_RA_KNRI", "FFI_RA_MNRR",
"FFI_RA_MORU", "FFI_RA_SCBL", "FFI_RA_THRO", "FFI_RA_WICA"),
export_tables = T, export_path = "C:/temp/")
Export views to zip
Even better, is exporting the views into a zip file of csvs. To do
that, add export_views = T. If no export_path is specified,
the zip file will be saved in your working directory. The file name and
path of the zip file will be printed in your console. This zip file is
then the fastest way to import FFI data into R using the
importViews() function instead.
importData(type = 'local',
dbname = c("FFI_RA_AGFO", "FFI_RA_BADL", "FFI_RA_DETO", "FFI_RA_FOLA",
"FFI_RA_FOUS", "FFI_RA_JECA", "FFI_RA_KNRI", "FFI_RA_MNRR",
"FFI_RA_MORU", "FFI_RA_SCBL", "FFI_RA_THRO", "FFI_RA_WICA"),
export_views = T, export_path = "C:/temp")
Return raw tables too
There are times when it helps to view the raw FFI data tables, such
as for troubleshooting an issue in the R package, or in figuring out
where an error in the data was introduced and need to be fixed. To
return the raw FFI tables in addition to the views, specify
keep_tables = TRUE.
importData(type = 'local',
dbname = c("FFI_RA_AGFO", "FFI_RA_BADL", "FFI_RA_DETO", "FFI_RA_FOLA",
"FFI_RA_FOUS", "FFI_RA_JECA", "FFI_RA_KNRI", "FFI_RA_MNRR",
"FFI_RA_MORU", "FFI_RA_SCBL", "FFI_RA_THRO", "FFI_RA_WICA"),
keep_tables = T)
Return views to global environment
importData(type = 'local',
dbname = c("FFI_RA_AGFO", "FFI_RA_BADL", "FFI_RA_DETO", "FFI_RA_FOLA",
"FFI_RA_FOUS", "FFI_RA_JECA", "FFI_RA_KNRI", "FFI_RA_MNRR",
"FFI_RA_MORU", "FFI_RA_SCBL", "FFI_RA_THRO", "FFI_RA_WICA"),
new_env = F)
The functions in the plantcomNGPN package are designed
to work with the views in VIEWS_NGPN, and are the best way to
interact with the data to query by park, site, site type, year,
parameter, etc. However, if you want to look at the views without using
the package functions, you can use the following code.
# See list of the views
names(VIEWS_NGPN)
## [1] "Disturbance_History"
## [2] "Cover_Points_metric"
## [3] "Trees_metric"
## [4] "SampleEvents"
## [5] "Density_Belts_metric"
## [6] "Surface_Fuels_Fine"
## [7] "Surface_Fuels_1000Hr"
## [8] "Density_Quadrats_metric"
## [9] "Cover_Species_Composition"
## [10] "Taxa_Table"
## [11] "Surface_Fuels_Duff"
## [12] "MacroPlots"
# Inspect the cover point view
View(VIEWS_NGPN$Cover_Points_metric)
#head(VIEWS_NGPN$Cover_Points_metric)
# Assign the coverpoint view to the object covpt, which now behaves like a data.frame.
covpt <- VIEWS_NGPN$Cover_Points_metric
Additionally, if you want to view the raw FFI data tables, and you imported the data into the NPGN_tables environment, you can access them with the code below:
importData(type = 'local',
dbname = c("FFI_RA_AGFO", "FFI_RA_BADL", "FFI_RA_DETO", "FFI_RA_FOLA",
"FFI_RA_FOUS", "FFI_RA_JECA", "FFI_RA_KNRI", "FFI_RA_MNRR",
"FFI_RA_MORU", "FFI_RA_SCBL", "FFI_RA_THRO", "FFI_RA_WICA"),
keep_tables = T)
# See list of FFI tables
sort(names(NGPN_tables))
# Inspect the MacroPlot table
View(NGPN_tables$MacroPlot)
# Assign the MacroPlot table to the object macro
macro <- NGPN_tables$MacroPlot
The functions in plantcomNGPN have help documentation
like any R package. To view the help, you can go to the Packages tab and
click on plantcomNGPN (see below). That will show you all the functions
in the package. Clicking on individual functions will take you to the
help documentation for that function.
You can also see the help of a function by running, for example:
?importData
If plantcomNGPN isn’t loaded yet, you’d run:
?plantcomNGPN::importData
Each function’s help includes a Description, Usage (i.e. function arguments and their defaults), Argument options/definitions, and several examples showing how the function can be used.
This is where you come in! If you notice typos or can think of better descriptions, examples, error messages, etc., please send them my way! After we’re more comfortable with R packages and get versed on GitHub, you’ll be able to make those changes directly in the package. For now, you can just send me your suggestions and I’ll make the changes.
Finally, if you ever want to peak under the hood at the function, you can view it several ways.View code in the GitHub katemmiller/plantcomNGPN repo. The functions are in the R folder.